Full-spectrum flash for electronic devices

ABSTRACT

Introduced here are light sources for flash photography configured to produce high-fidelity white light that is tunable over a broader range of correlated color temperatures (CCTs) than conventional flash technologies. The light source can include multiple independently controllable color channels representing illuminants (e.g., light-emitting diodes) of different colors with varying degrees of saturation. Operating collectively, the multiple color channels can produce a high spectral quality white light corresponding to different CCTs (e.g., “warm” white light having a red hue, “cool” white light having a blue hue). Operating independently, these same color channels can be pre-flashed in a variety of prescribed sequences to probe the spectral characteristics of a scene, thereby allowing for an enhanced, spectrally matched white flash as well as collecting per-pixel reflectivity data that can be later used in during post processing of the captured image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.16/030,679, titled “Full-Spectrum Flash For Electronic Devices” andfiled on Jul. 9, 2018, which claims priority to U.S. ProvisionalApplication No. 62/530,244, titled “Multi-Channel Full Color SpectrumFlash for Mobile Devices” and filed on Jul. 9, 2017, which areincorporated by reference herein in their entirety.

TECHNICAL FIELD

Various embodiments concern multi-channel light sources that are tunableacross a wide color gamut and capable of producing full-spectrum flash.

BACKGROUND

Traditional lighting technologies, such as incandescent bulbs andfluorescent bulbs, suffer from several drawbacks. For example, theselighting technologies do not have long lifespans or high energyefficiencies. Moreover, these lighting technologies are only offered ina limited selection of colors, and the light output by these lightingtechnologies generally changes over time as the source ages and beginsto degrade. Consequently, light-emitting diodes (LEDs) have become anattractive option for many applications.

Many electronic devices include one or more image sensors for capturingimages of the surrounding environment, such as a rear-facing camera or afront-facing camera. Each of these cameras is typically accompanied byat least one illuminant capable of providing robust luminosity across awide field of view (FOV). Yet these illuminants are typically deficientin several respects.

For instance, LEDs embedded within electronic devices are often designedto produce a fixed white light with no tunable range. White light couldbe produced by combining a short-wavelength LED (e.g., one designed toproduce blue light or ultraviolet light) and a yellow phosphor coating.Blue/ultraviolet photons generated by the short-wavelength LED willeither travel through the phosphor layer without alteration or beconverted into yellow photons in the phosphor layer. The combination ofblue/ultraviolet photons and yellow photons produces white light (alsoreferred to as “phosphor white light”). As another example, white lightcould be produced by a xenon flashlamp designed to produce extremelyintense white light (also referred to as “xenon white light”) for shortdurations.

When an image is captured by an electronic device under phosphor whitelight or xenon white light, the effect is roughly equivalent tocapturing the image under a fluorescent light source. Thus, phosphorwhite light and xenon white light will not provide an accuratelyreflected color spectrum, nor will they have any vibrancy. Instead,these illuminants simply flood the ambient environment with white lightso that objects can be readily identified within images.

Recent development has focused on developing light sources that includetwo illuminants corresponding to different correlated color temperatures(CCTs). While these light sources may be able to produce a mixed whitelight that more accurately matches the color of an ambient environment,they can further take away from the color quality. For example, whenmixed white light drops below the Planckian locus (also referred to asthe “black body locus”) it may become pinkish in tone. Consequently,significant post-processing may be necessary to artificially recreatethe original lighting of the ambient environment.

BRIEF DESCRIPTION OF THE DRAWINGS

Various features of the technology will become more apparent to thoseskilled in the art from a study of the Detailed Description inconjunction with the drawings. Embodiments of the technology areillustrated by way of example and not limitation in the drawings, inwhich like references may indicate similar elements.

This application contains at least one drawing executed in color. Copiesof this application with color drawing(s) will be provided by the Officeupon request and payment of the necessary fees.

FIG. 1A depicts a top view of a multi-channel light source that includesmultiple color channels that are configured to produce different colors.

FIG. 1B depicts a side view of the multi-channel light sourceillustrating how, in some embodiments, the illuminants can reside withina housing.

FIG. 1C depicts an electronic device that includes a rear-facing cameraand a multi-channel light source configured to illuminate the ambientenvironment.

FIG. 2 depicts an example of an array of illuminants (here,light-emitting diodes).

FIG. 3A illustrates the tunable range of a two-channel light source incomparison to the Planckian locus (also referred to as the “black bodylocus”).

FIG. 3B illustrates the tunable range of a five-channel light source.

FIG. 4 illustrates the visual impact of Duv on images captured inconjunction with flashes of white light produced by a two-channel lightsource and a five-channel light source.

FIG. 5 illustrates how the human eye of an average individual willgenerally recognize improvements in color reproducibility (i.e., asmeasured in terms of R_(f) and R_(g) values).

FIG. 6A depicts average ΔE of all color bins for four different types oflight source: conventional flash technology for mobile phones; atwo-channel light source at 5000K; a two-channel light source at 2700K;and a five-channel light source as described herein.

FIG. 6B depicts ΔE by surface type for four different types of lightsource: conventional flash technology for mobile phones; a two-channellight source at 5000K; a two-channel light source at 2700K; and afive-channel light source as described herein.

FIG. 7 depicts two different color properties (i.e., CRI and R9) for twodifferent types of illuminant: a two-channel light source and afive-channel light source.

FIGS. 8A-D illustrate the ability of four different types of lightsource to mimic the visible spectrum of an ambient scene.

FIGS. 9A-D illustrate the ability of the four different types of lightsource to properly mimic chromaticity of an ambient scene.

FIG. 10 illustrates the total achievable color gamut of a five-channellight source in comparison to a conventional chromaticity diagram.

FIG. 11 illustrates how the five-channel light sources described hereincan substantially improve in terms of color reproducibility incomparison to two-channel light sources.

FIG. 12 illustrates a process for acquiring color information that maybe useful in tuning each color channel of a five-channel light source inpreparation for a flash event.

FIG. 13 illustrates another process for acquiring color information thatmay be useful in tuning each color channel of a five-channel lightsource in preparation for a flash event.

FIG. 14 illustrates a process for performing a processing procedure onimages captured by an electronic device.

FIG. 15 depicts a fidelity comparison between the five-channel lightsource and the built-in flash technology of a Huawei® Nexus 6P mobilephone.

FIG. 16 depicts a fidelity comparison between the five-channel lightsource and the built-in flash technology of a Google Pixel™ mobilephone.

FIG. 17 depicts a fidelity comparison between the five-channel lightsource and the built-in flash technology of a Samsung® Galaxy mobilephone.

FIG. 18 depicts a fidelity comparison between the five-channel lightsource and the built-in flash technology of an Apple iPhone® 7 mobilephone.

FIG. 19 depicts a fidelity comparison between the five-channel lightsource and the built-in flash technology of an Apple iPhone® X mobilephone.

FIG. 20 is a block diagram illustrating an example of a processingsystem in which at least some operations described herein can beimplemented.

The drawings depict various embodiments for the purpose of illustrationonly. Those skilled in the art will recognize that alternativeembodiments may be employed without departing from the principles of thetechnology. Accordingly, while specific embodiments are shown in thedrawings, the technology is amenable to various modifications.

DETAILED DESCRIPTION

An illuminant can be characterized by its color temperature and colorrendering index (CRI). The color temperature of an illuminant is thetemperature at which the color of light emitted from a heated black bodyis matched by the color of the illuminant. For an illuminant that doesnot substantially emulate a black body, such as a fluorescent bulb or alight-emitting diode (LED), the correlated color temperature (CCT) ofthe illuminant is the temperature at which the color of light emittedfrom a heated black body is approximated by the color of the illuminant.

CCT can also be used to represent the chromaticity of illuminants thatare configured to generate white light. Because chromaticity is atwo-dimensional characterization, Duv (as defined by the AmericanNational Standards Institute (ANSI) C78.377) may be used to provideanother dimension. When used with a MacAdam ellipse, CCT and Duv allowthe visible color output by an illuminant to be more preciselycontrolled (e.g., by being tuned). The MacAdam ellipse represents allcolors that are distinguishable to the human eye.

CRI, which is measured on a scale from 1-100, indicates how accuratelyan illuminant renders the color(s) of an illuminated object incomparison to an ideal or natural light source. If the illuminant is anLED, CRI is calculated by measuring the CRI ratings for eight differentcolors (i.e., R1-R8) and then averaging them out. However, the measuredaverage fails to account for R9 (i.e., red) and R13 (i.e., skin tone),which are often useful in mixing/reproducing the other colors (e.g., tomake skin tones appear more natural).

The CCT and CRI of an illuminant is typically difficult to tune,particularly in real time (e.g., as an image of a scene that isilluminated by the illuminant is captured). Further difficulty ariseswhen trying to maintain an acceptable CRI while varying the CCT of anilluminant.

Introduced here, therefore, are multi-channel light sources configuredto produce high-fidelity white light that is tunable over a broader CCTrange than conventional flash technologies, such as phosphor LEDs andxenon flashlamps. A multi-channel light source includes multiple colorchannels corresponding to different colors, and each color channel caninclude one or more illuminants configured to produce a substantiallysimilar color. One example of an illuminant is an LED.

A controller can be configured to simultaneously drive each colorchannel of the multiple color channels to produce a white light having adesired CCT. The controller may be configured to determine, based on acolor mixing model, one or more operating parameter(s) required toachieve the desired CCT. For example, the operating parameter(s) mayspecify the driving current to be provided to each color channel. Byvarying the operating parameter(s), the controller can tune the CCT ofthe white light as necessary. Accordingly, the multi-channel lightsource can produce white lights corresponding to different CCTs (e.g.,“warm” white light having a red hue, “cool” white light having a bluehue). White light produced by these multi-channel light sources canimprove the quality of images taken in the context of consumerphotography, prosumer photography, professional photography, etc.

A multi-channel light source can include, for example, fivestrategically selected, saturated color channels that synergisticallyoverlap across the visible range.

When these color channels are combined, the multi-channel light sourcecan exhibit several advantages including:

-   -   The ability to reproduce nearly all real white lights, both        natural and artificial, to a level yielding near-zero color        distortion (e.g., ΔE<1) for an arbitrary multi-channel image        sensor. Near-zero color distortion can be achieved for all real        colors from grays to fully saturated colors. CCT, tint, spectral        profile, and response profile of the multi-channel image sensor        may all be integral to the illuminance spectrum.    -   The ability to produce significantly greater illumination per        strobe aperture/die area than conventional multi-channel white        light sources and monochromatic light sources due to greater die        area utilization.    -   The ability to eliminate perceptible irreversible metamerisms        introduced after standard chromatic adaptations of images        captured in conjunction with light produced by state-of-the-art        strobes with fully known spectra.    -   The ability to provide a universal strobe with relaxed color        channel binning requirements compared to conventional white        strobes. A “universal” strobe may be able to readily adapt to        multi-channel image sensor response variations.

When these color channels are strobed in concert with a multi-channelimage sensor (e.g., an RGB camera sensor), the multi-channel lightsource can exhibit several advantages including:

-   -   The ability to precisely reveal the universal visible-range        spectral reflectivity profile of all illuminated surfaces on a        per-pixel basis. Said another way, underlying surface        reflectivity can be revealed rather than simply the apparent        color (e.g., in an RGB sense).    -   Per-pixel reflectivity in turn enables:        -   Spectral identification of scene illuminant;        -   Identification of arbitrary multiple scene illuminants            (e.g., an indoor source and an outdoor source via a window,            scene-influenced illuminant tints, etc.);        -   Pixel-level illuminant spectrum identification;        -   True color of each pixel under any known illuminant, rather            than just the apparent pixel color; and        -   Non-perceptible error chromatic adaptation for all colors.    -   Accordingly, the multi-channel light source also provides:        -   The ability to perfectly re-cast strobe-lit portions of a            scene as if lit by native scene lighting spectra and            relative intensities as the scene appears to the human eye.            This results in “lifting” otherwise dark, noisy, or blurry            images to be sufficiently lit, non-noisy, or non-blurry, yet            have the appearance of no flash.        -   An accurate-to-human-perception color re-casting of            arbitrary multi-illuminant scenes.        -   A controllable strobe able to provide image enhancement. As            such, the multi-channel light source can be more            ubiquitously employed than conventional flash technologies            with inferior results.        -   Computer software (e.g., mobile application) utility. For            example, an individual may be able to modify/monitor            spectral reflective level (e.g., by matching the color of a            painted surface, section of fabric, etc.) to match color            across a broad color gamut, spectral assessments, image            sensor response profiles, etc.

Embodiments may be described with reference to particular electronicdevices or illuminants. For example, the technology may be described inthe context of mobile phones that include a multi-channel light sourcehaving LEDs of several different colors. However, those skilled in theart will recognize that these features are equally applicable to othertypes of electronic devices and illuminants.

Moreover, the technology can be embodied using special-purpose hardware(e.g., circuitry), programmable circuitry appropriately programmed withsoftware and/or firmware, or a combination of special-purpose hardwareand programmable circuitry. Accordingly, embodiments may include amachine-readable medium having instructions that may be used to programa light source to perform a process for controllably producing whitelight (e.g., in the form of a flash) having a high gamut area by mixingthe colored light produced by multiple color channels.

Terminology

References in this description to “an embodiment” or “one embodiment”means that the particular feature, function, structure, orcharacteristic being described is included in at least one embodiment.Occurrences of such phrases do not necessarily refer to the sameembodiment, nor are they necessarily referring to alternativeembodiments that are mutually exclusive of one another.

Unless the context clearly requires otherwise, the words “comprise” and“comprising” are to be construed in an inclusive sense rather than anexclusive or exhaustive sense (i.e., in the sense of “including but notlimited to”). The terms “connected,” “coupled,” or any variant thereofis intended to include any connection or coupling between two or moreelements, either direct or indirect. The coupling/connection can bephysical, logical, or a combination thereof. For example, objects may beelectrically or communicatively coupled to one another despite notsharing a physical connection.

The term “based on” is also to be construed in an inclusive sense ratherthan an exclusive or exhaustive sense. Thus, unless otherwise noted, theterm “based on” is intended to mean “based at least in part on.”

The term “module” refers broadly to software components, hardwarecomponents, and/or firmware components. Modules are typically functionalcomponents that can generate useful data or other output(s) based onspecified input(s). A module may be self-contained. A computer programmay include one or more modules. Thus, a computer program stored in amemory accessible to a multi-channel light source may include multiplemodules responsible for completing different tasks or a single moduleresponsible for completing all tasks.

When used in reference to a list of multiple items, the word “or” isintended to cover all of the following interpretations: any of the itemsin the list, all of the items in the list, and any combination of itemsin the list.

The sequences of steps performed in any of the processes described hereare exemplary. However, unless contrary to physical possibility, thesteps may be performed in various sequences and combinations. Forexample, steps could be added to, or removed from, the processesdescribed here. Similarly, steps could be replaced or reordered. Thus,descriptions of any processes are intended to be open-ended.

Light Source Overview

FIG. 1A depicts a top view of a multi-channel light source 100 thatincludes multiple color channels that are configured to producedifferent colors. Each color channel can include one or more illuminants102 of a substantially similar color. For example, the multi-channellight source 100 may include a single illuminant configured to produce afirst color, multiple illuminants configured to produce a second color,etc. Note that, for the purpose of simplification, a color channel maybe said to have “an illuminant” regardless of how many separateilluminants the color channel includes.

One example of an illuminant is an LED. An LED is a two-lead illuminantthat is generally comprised of an inorganic semiconductor material.While embodiments may be described in the context of LEDs, thetechnology is equally applicable to other types of illuminant. Table Iincludes several examples of available colors of LEDs, as well as thecorresponding wavelength range, voltage drop, and semiconductormaterial(s).

TABLE I Range (in nanometers) in which the dominant wavelength resides,voltage drop, and representative materials for available colors of LEDs.Color Dominant Wavelength Representative Materials Infrared λ > 760Gallium arsenide; and Aluminum gallium arsenide Red 610 < λ < 760Aluminum gallium arsenide; Gallium arsenide phosphide; Aluminum galliumindium phosphide; and Gallium(III) phosphide Orange 590 < λ < 610Gallium arsenide phosphide; Aluminum gallium indium phosphide; andGallium(III) phosphide Yellow 570 < λ < 590 Gallium arsenide phosphide;Aluminum gallium indium phosphide; and Gallium(III) phosphide Green 500< λ < 570 Aluminum gallium phosphide; Aluminum gallium indium phosphide;Gallium(III) phosphide; Indium gallium nitride; and Gallium(III) nitrideBlue 450 < λ < 500 Zinc selenide; and Indium gallium nitride Violet 400< λ < 450 Indium gallium nitride Ultraviolet λ < 400 Indium galliumnitride; Diamond; Boron nitride; Aluminum nitride; Aluminum galliumnitride; and Aluminum gallium indium nitride

Other colors not shown in Table I may also be incorporated into thelight source 100. Examples of such colors include cyan (490<λ<515), lime(560<λ<575), amber (580<λ<590), and indigo (425<λ<450). Those skilled inthe art will recognize that these wavelength ranges are simply includedfor the purpose of illustration.

As noted above, a multi-channel light source 100 includes multiple colorchannels that are configured to produce different colors. For example,the light source 100 may include three separate color channelsconfigured to produce blue light, green light, and red light. Such lightsources may be referred to as “RGB light sources.” As another example,the light source 100 may include four separate color channels configuredto produce blue light, green light, red light, and either amber light orwhite light. Such light sources may be referred to as “RGBA lightsources” or “RGBW light sources.” As another example, the light source100 may include five separate color channels configured to produce bluelight, cyan light, lime light, amber light, and red light. As anotherexample, the light source 100 may include seven separate color channelsconfigured to produce blue light, cyan light, green light, amber light,red light, violet light, and white light. Thus, the light source 100could include three channels, four channels, five channels, sevenchannels, etc.

While three-channel light sources and four-channel light sources improveupon conventional flash technologies, they may have a lumpy spectraldistribution or narrow range of high fidelity. Consequently, themulti-channel light source 100 will often include at least fivedifferent color channels. As the number of color channels increases, thelight quality, CCT range, and quality over range will also generallyincrease. For example, a five-channel light source having properlyselected illuminants can be designed to deliver full-spectrum whitelight over a broad CCT range (e.g., from 1650K to over 10000K) withCRI/R9 values greater than 90 (e.g., with a typical Ra average value of94) at ΔuV of ±0.002. Multi-channel light sources able to a color errorof less than one (i.e., ΔE<1) may be referred to as “high-fidelity lightsources.” Gradients of color could also be added to match the tint of ascene.

Due to their low heat production, LEDs can be located close together.Thus, if the illuminants 102 of the multi-channel light source are LEDs,then the light source 100 may include an array comprised of multipledies placed arbitrarily close together. Note, however, that theplacement may be limited by “whitewall” space between adjacent dies. Thewhitewall space is generally on the order of approximately 0.1millimeters (mm), though it may be limited (e.g., no more than 0.2 mm)based on the desired diameter of the light source 100 as a whole. InFIG. 2, for example, the array includes eight dies associated with fivedifferent colors Such an array may be sized to fit within similardimensions as conventional flash technology. The array may also be basedon standard production die(s) requiring, for example, a 2-1-1-0.5-0.5area ratio of lime-amber-cyan-red-blue. The array may be driven by oneor more linear field-effect transistor-based (FET-based)current-regulated drivers. In some embodiments, each color channel isconfigured to be driven by a corresponding driver. These drivers may beaffixed to, or embedded within, a substrate 104 arranged beneath theilluminants 102.

By controllably driving each color channel on an individual basis, themulti-channel light source 100 can produce white light at differentCCTs. For example, the multi-channel light source 100 may produce ahigh-fidelity flash in conjunction with the capture of an image by anelectronic device. Said another way, the light source 100 may generate ahigh-fidelity flash to illuminate the scene being photographed by anelectronic device. Examples of electronic devices include mobile phones,tablet computers, digital cameras (e.g., single-lens reflex (SLR)cameras, digital SLR (DSLR) cameras, and light-field cameras, which mayalso be referred to as “plenoptic cameras”), etc. White light producedby the multi-channel light source 100 can improve the quality of imagestaken in the context of consumer photography, prosumer photography,professional photography, etc.

Although the illuminants 102 are illustrated as an array of LEDspositioned on a substrate 104, other arrangements are also possible. Insome cases, a different arrangement may be preferred (e.g., due tothermal constraints, size constraints, color mixing constraints, etc.).For example, the multi-channel light source 100 may include a circulararrangement, grid arrangement, or cluster of LEDs.

In some embodiments, the multi-channel light source 100 iscommunicatively coupled to an optical sensor (e.g., a photodiode)configured to generate optical feedback indicative of the brightnesslevel of each color channel. For example, the multi-channel light source100 may include multiple optical sensors corresponding to differentbandwidths. In such embodiments, the optical feedback may specifymultiple light intensities corresponding to optical sensors designed toexamine different bandwidths, thereby allowing color shift in a colorchannel to be readily discovered. As another example, the light source100 may include multiple optical sensors, and each optical sensor may beconfigured to measure the brightness of a corresponding color channel.The optical sensor(s) may be arranged along the upper surface of thesubstrate 104.

The multi-channel light source 100 may also include a heat sensor (e.g.,a thermistor) configured to measure thermal feedback. The heat sensormay measure thermal feedback periodically to determine whether theoperating temperature of the LED's aging has affected the output of theilluminants 102. Similar to the optical sensor(s), the heat sensor maybe arranged along the upper surface of the substrate 104.

A multivariate state estimator may be responsible for thermal managementof the multi-channel light source 100 may be executed by a controller(e.g., a processor) that is communicatively coupled to the multi-channellight source 100. Proper thermal management may be critical for flashevents due to their dynamic nature. In some embodiments the multivariatestate estimator resides within a memory of the multi-channel lightsource 100, while in other embodiments the multivariate state estimatorresides within a memory of an electronic device that is communicativelycoupled to the multi-channel light source 100. As further describedbelow, in some embodiments the multi-channel light source 100 resideswithin the electronic device, while in other embodiments themulti-channel light source is connected to the electronic device acrossa network. The multivariate state estimator may be configured toestimate the thermal state of the light source 100 based on thermalinteractions of the ambient temperature and adjacent die.

An empirical solver module (also referred to as a “characterizationmodule” or a “fast inverse solver”) can also be configured tocharacterize light emitted by the light source 100. The characterizationmodule can implement the processes (e.g., the inverse solver algorithm)described in U.S. application Ser. No. 15/425,467, which is incorporatedby reference herein in its entirety. Thus, the characterization modulemay be configured to determine, based on a reference set of curvescorresponding to different combinations of operating conditions (e.g.,driving current and flux), an appropriate brightness and color point(s)necessary to achieve a particular color model corresponding to aparticular CCT.

FIG. 1B depicts a side view of the multi-channel light source 100illustrating how, in some embodiments, the illuminants 102 can residewithin a housing. The housing can include a base plate 106 thatsurrounds the illuminants 102 and/or a protective surface 108 thatcovers the illuminants 102. While the protective surface 108 shown hereis in the form of a dome, those skilled in the art will recognize thatother designs are possible. For example, the protective surface 108 mayinstead be arranged in parallel relation to the substrate 104. Moreover,the protective surface 108 may be designed such that, when themulti-channel light source 100 is secured within an electronic device,the upper surface of the protective surface 108 is substantiallyco-planar with the exterior surface of the electronic device. Theprotective substrate 108 can be comprised of a material that issubstantially transparent, such as glass, plastic, etc.

The substrate 104 can be comprised of any material able to suitablydissipate heat generated by the illuminants 102. A non-metal substrate,such as one comprised of woven fiberglass cloth with an epoxy resinbinder (e.g., FR4), may be used to reduce/eliminate the problemsassociated with metal substrates. For example, a substrate 104 composedof FR4 can more efficiently dissipate the heat generated by multiplecolor channels without experiencing the heat retention issues typicallyencountered by metal substrates. Note, however, that some non-metalsubstrates can only be used in combination with mid-power illuminants(i.e., rather than high-power illuminants). High-power illuminants(e.g., those useful for producing flashes in the context of photography)may instead be mounted on a substrate 104 comprised of metal, ceramic,etc.

The processing components necessary for operating the illuminants 102may be physically decoupled from the light source 100. For example, theprocessing components may be connected to the illuminants 102 viaconductive wires running through the substrate 104. Examples ofprocessing components include drivers 110, controllers 112 (e.g.,processors), power sources 114 (e.g., batteries), etc. Consequently, theprocessing components need not be located within the light source 100.Instead, the processing components may be located elsewhere within theelectronic device within which the light source 100 is installed.Additional information on the decoupling of processing components from alight source can be found in U.S. application Ser. No. 15/382,575, whichis incorporated by reference herein in its entirety.

As noted above, the multi-channel light source 100 may be designed tooperate in conjunction with an image sensor. An image sensor is a sensorthat detects information that constitutes an image. Generally, an imagesensor does so by converting the variable attenuation of light waves(e.g., as they pass through or reflect off of objects) into electricalsignals, which represent small bursts of current that convey theinformation. Examples of image sensors includesemiconductor-charge-coupled devices (CCDs) and complementarymetal-oxide-semiconductor sensors (CMOS) sensors. Both types of imagesensor accomplish the same task (i.e., capture light and convert it intoelectrical signals). However, because CMOS sensors are generallycheaper, smaller, and consume less power than CCDs, many electronicdevices (e.g., mobile phones) use CMOS sensors for image capture.

Accordingly, the multi-channel light source 100 could be configured togenerate a flash responsive to determining that an image sensor hasreceived an instruction to capture an image of a scene. The instructionmay be created responsive to receiving input indicative of user inputrequesting that the image be captured. As shown in FIG. 1C, an imagesensor (here, a camera 152) may be housed within the same electronicdevice as a multi-channel light source. The user input may be in theform of tactile input provided along the surface of a touch-sensitivedisplay or a mechanical button accessible along the exterior of theelectronic device.

In some embodiments, the multi-channel light source is designed suchthat it can be readily installed within the housing of an electronicdevice. FIG. 1C depicts an electronic device 150 that includes arear-facing camera 152 and a multi-channel light source 154 configuredto illuminate the ambient environment. The multi-channel light source154 may be, for example, the multi-channel light source 100 of FIGS.1A-B. The rear-facing camera 152 is one example of an image sensor thatmay be configured to capture images in conjunction with light producedby the light source 100. Here, the electronic device 150 is a mobilephone. However, those skilled in the art will recognize that thetechnology described herein could be readily adapted for other types ofelectronic devices, such as tablet computers and digital cameras.

The camera 152 is typically one of multiple image sensors included inthe electronic device 150. For example, the electronic device 100 mayinclude a front-facing camera that allows an individual to capture stillimages or video while looking at the display. The rear-facing andfront-facing cameras can be, and often are, different types of imagesensors that are intended for different uses. For example, the imagesensors may be capable of capturing images having different resolutions.As another example, the image sensors could be paired with differentlight sources (e.g., the rear-facing camera may be associated with astronger flash than the front-facing camera, or the rear-facing cameramay be disposed in proximity to a multi-channel light source while thefront-facing camera is disposed in proximity to a single-channel lightsource).

Other components may also be disposed along the exterior of the housing158 of the electronic device 150. For example, a microphone 156 can beconfigured to generate audio data when other actions are performed(e.g., an image is captured, a call is placed, etc.). The audio data maybe used for noise cancellation purposes (e.g., to reduce ambient noisein video media generated by the camera 152).

FIG. 2 depicts an example of an array 200 of illuminants 202. If theilluminants 202 are LEDs, the array 200 may be produced using standarddies (also referred to as “chips”). A die is a small block ofsemiconducting material on which the diode is located. Typically, diodescorresponding to a given color are produced in large batches on a singlewafer (e.g., comprised of electronic-grade silicon, gallium arsenide,etc.), and the wafer is then cut (“diced”) into many pieces, each ofwhich includes a single diode. Each of these pieces may be referred toas a “die.”

As shown in FIG. 2, the array 200 includes multiple color channelsconfigured to produce light of different colors. Here, for example, thearray 200 includes five color channels (i.e., blue, cyan, lime, amber,and red). Each color channel can include one or more illuminants. Here,for example, three color channels (i.e., blue, lime, and red) includemultiple illuminants, while two color channels (i.e., cyan and amber)include a single illuminant. The number of illuminants in each colorchannel, as well as the arrangement of these illuminates within thearray 200, may vary based on the desired output characteristics, such asmaximum CCT, minimum CCT, maximum temperature, etc.

The array 200 is generally capable of producing light greater than 1,000lumens, though some embodiments are designed to produce light less than1,000 lumens (e.g., 700-800 lumens during a flash event). In someembodiments, the illuminants 202 are positioned in the array 200 in ahighly symmetrical pattern to improve spatial color uniformity. Forexample, when the array 200 is designed to produce white light throughsimultaneous driving of the multiple color channels, the illuminantscorresponding to those color channels may be arranged symmetrically tofacilitate mixing of the colored light.

The array 200 may be designed such that it can be installed within thehousing of an electronic device (e.g., electronic device 150 of FIG. 1C)in addition to, or instead of, a conventional flash component. Forexample, some arrays designed for installation within mobile phones areless than 4 mm in diameter, while other arrays designed for installationwithin mobile phones are less than 3 mm in diameter. The array 200 mayalso be less than 1 mm in height. In some embodiments, the totalestimated area necessary for the array may be less than 3 mm² prior toinstallation and less than 6 mm² after installation. Such a designenables the array 200 to be positioned within a mobile phone withoutrequiring significant repositioning of components within the mobilephone. One advantage of designing such a compact array of dies is thatit can achieve good color mixing and adequate field of view (FOV)without the use of a collimator, diffuser, or lens.

In some embodiments, the array 200 is positioned beneath a diffuserdesigned to ensure proper color mixing. In other embodiments, the array200 is positioned within a collimator 204 (also referred to as a “mixingpipe”) designed to ensure proper spatial color uniformity of lightproduced by the illuminants 202. The collimator 204 may also be designedto promote uniform color mixing and control the FOV of light emitted bythe array 200. The collimator 204 can be comprised of an inflexiblematerial (e.g., glass) or a flexible material (e.g., silicone). Thecollimator 204 may be in the form of a tubular body. In some embodimentsthe egress aperture of the tubular body is narrower than the array(e.g., the egress aperture may have a diameter of 2.5 mm, 3 mm, or 3.5mm), while in other embodiments the egress aperture of the tubular bodyis wider than the array (e.g., the egress aperture may have a diameterof 4.5 mm, 5 mm, or 5.5 mm). Thus, the tubular body may have a slopedinner surface that either focuses or disperses light produced by theilluminants 202.

The array 200 may be used instead of, or in addition to, conventionalflash technologies that are configured to generate a flash inconjunction with the capture of an image. Thus, an electronic device(e.g., electronic device 150 of FIG. 1C) may include a single-channellight source and/or a multi-channel light source.

Application to Flash Events

Technologies for virtually binning illuminants, as well as usingthermal/optical feedback mechanisms for tuning illuminants, have createdan opportunity to use multi-channel, full-spectrum light sources tocreate the white light necessary for a flash event. Additionalinformation on thermal/optical feedback mechanisms can be found in U.S.application Ser. No. 15/382,578, which is incorporated by referenceherein in its entirety. Additional information on provisioning colormixing models can be found in U.S. application Ser. No. 15/609,619,which is incorporated by reference herein in its entirety.

Moreover, the improved geometries of illuminants (e.g., colored diodes)and dies on which these illuminants are located have permitted theassembly of light sources that include multiple color channels. Suchadvancements permit the construction of multi-channel, addressablearrays capable of fitting within the space presently allocated to theconventional flash component in electronic devices. Such a design canoffer the possibility of professional-level output (also referred to as“stage-lighting-quality output”) at over 1,000 lumens in an array havinga diameter of approximately 3-4 mm. Moreover, such a design can offerthe ability to provide broad-spectrum flashes while also providing thebenefits of having individually addressable color channels in the gamutarea.

In comparison to convention flash technologies, the multi-channel lightsources described herein offer several benefits, including:

-   -   A true color match (i.e., ΔE of less than one) over a broad CCT        range (e.g., from 1650K to over 10000K). ΔE (also referred to as        the “color error” or the “unity error”) is a measure of change        in visual perception between two colors. ΔE values of less than        one are not perceptible to the human eye.    -   Minimal Duv error (e.g., ΔuV<0.002).    -   A higher, flatter flux over the broad CCT range.    -   A higher flux-to-aperture ratio.    -   A lower peak power draw.    -   Access to a wider color gamut than other technologies.

FIG. 3A illustrates the tunable range of a two-channel light source incomparison to the Planckian locus (also referred to as the “black bodylocus”). The tunable range of two-channel light sources are typicallynarrow. For example, the tunable range of a two-channel light sourcewill often be approximately 2500-5000K, though it may expand to2700-6500K at its broadest. As shown in FIG. 3A, the light path betweenthe two color channels is substantially linear. The largest Duv,meanwhile, is in the middle of the tunable range.

FIG. 3B illustrates the tunable range of a five-channel light source.The tunable range of the five-channel light source is much larger thanthe tunable range of the two-channel light source. Here, for example,the tunable range is approximately 1650-8000K, though it may beoptimized to an even wider range for flash events. Rather than “walk” alinear path, the light path can instead walk along the Planckian locusto approximately 4500K and then follow the daylight locus toapproximately 8000K. Consequently, the five-channel light source canmore accurately reproduce both “warm” white lights having reddish hues(e.g., those intended to mimic candlelight, sunsets, etc.) and “cool”white lights having bluish hues (e.g., those intended to mimic blue sky,shade, etc.).

While embodiments may be described in the context of five-channel lightsources, those skilled in the art will recognize that the technology isequally applicable to three-channel light sources, four-channel lightsources, seven-channel light sources, etc. As the number of colorchannels increases, the ability of the light source to accuratelyproduce a desired CCT will also generally increase. Thus, aseven-channel light source may be able to more accurately produce agiven CCT than a five-channel light source or a three-channel lightsource, though the additional design complexities and illuminant costmay not necessarily be worthwhile. For example, because a five-channellight source can achieve a ΔE of less than one, additional colorchannels will often not be useful or noticeable. As the number of colorchannels increases, the total tunable range of the light source may alsogenerally increase.

FIG. 4 illustrates the visual impact of Duv on images captured inconjunction with flashes of white light produced by a two-channel lightsource and a five-channel light source. As noted above, Duv may be usedto describe the distance from the Planckian locus when examining thechromaticity of white light.

For the two-channel light sources shown here, Duv is approximately 0.008in the middle of the tuning range (though the actual value will dependon the CCTs of the two channels). For context, Duv values greater thanapproximately 0.002 are usually detectable. Phosphor-converted whitelight having low CRI will also typically result in desaturation. Thetomatoes on the left and bottom of FIG. 4 were captured in conjunctionwith a two-channel light source. The tomato on the left has becomevisibly desaturated (also referred to as the “dullness” of an image),while the tomato on the bottom has become visibly discolored due to ahue shift.

For five-channel light sources, Duv will consistently be below 0.002.Five-channel light sources minimize Duv by traversing either thePlanckian locus or the daylight locus across the entire tunable range.The tomato in the middle of FIG. 4 was captured in conjunction with afive-channel light source.

Because each channel of a five-channel light source can be separatelydriven (e.g., by a controller), various image characteristics can bemodified in real time. For example, the five-channel light source mayproduce white light that increases saturation to produce more vibrantcolors, as shown by the tomato on the right of FIG. 4. As anotherexample, the five-channel light source may produce white light thatshifts hue, as shown by the tomatoes on the top and bottom of FIG. 4.Thus, a five-channel light source can illuminate an ambient environmentso that object(s) are imaged as they naturally appear to the human eye,as well as provide an option to intentionally produce discoloration(e.g., by altering saturation, hue, etc.).

FIG. 5 illustrates how the human eye of an average individual willgenerally recognize improvements in color reproducibility (i.e., asmeasured in terms of R_(f) and R_(g) values). Moreover, as shown here,many individuals prefer high-fidelity white light corresponding to ared-enhanced gamut. Such light may be referred to as “warm” white light.The multi-channel light sources described herein can be configured toproduce “warm” white light during a flash event, as well as enable flashcharacteristics to be readily modified (e.g., in real time or duringpost-processing). For example, the source controller may vary thecurrent driving each color channel to produce a modified white lightthat results in decreased saturation so that image(s) in conjunctionwith the modified white light will fall within the preferred segment ofthe spectrum shown in FIG. 5.

FIG. 6A depicts average ΔE of all color bins for four different types oflight source: conventional flash technology for mobile phones; atwo-channel light source at 5000K; a two-channel light source at 2700K;and a five-channel light source as described herein. Based on theIlluminating Engineering Society (IES) Technical Memorandum (TM) 30-15,ΔE results in a loss of image color information that is not recoverable(e.g., through post-processing). Ideally, ΔE (also referred to as “colorerror” or “unity error”) from an illuminant should be less than one.When comparison of two colors results in a ΔE value of less than one,the two colors are not distinguishable by the human eye.

As shown in FIG. 6A, the five-channel light sources described herein canconsistently achieve a ΔE of significantly less than one (e.g.,generally about 0.6-0.7). Accordingly, images captured in conjunctionwith a flash produced by a five-channel light source will moreaccurately reflect the actual colors of objects in the captured scene.

Two-channel, three-channel, and four-channel light sources can alsoimprove upon conventional flash technologies. While these light sourceswill have better color rendering properties than conventional flashtechnologies, they typically cannot achieve a ΔE of less than one.Consequently, light sources having at least five color channels may bepreferred in some instances.

FIG. 6B depicts ΔE by surface type for four different types of lightsource: conventional flash technology for mobile phones; a two-channellight source at 5000K; a two-channel light source at 2700K; and afive-channel light source as described herein. The results of examininga variety of different surface types (e.g., nature, skin, textiles,etc.) are shown here. Similar results were discovered for other surfacetypes.

For a given light source, ΔE will vary based on the characteristics ofthe surface being illuminated. For example, as shown in FIG. 6B, ΔE istypically higher for nature surfaces than skin surfaces. Thefive-channel light sources described herein can consistently producelower ΔE values than these other light sources, regardless of surfacetype. Moreover, these five-channel light sources can generally reducethe color loss below the visually distinguishable threshold (i.e., ΔE<1)with limited exceptions. Here, for example, ΔE associated with naturesurfaces may be slightly greater than one, though the five-channel lightsource still results in a much smaller ΔE than these other lightsources. Moreover, higher ΔE for nature surfaces is not likely to be asignificant issue as images of outdoor environments are generally notcaptured in conjunction with a flash.

FIG. 7 depicts two different color properties (i.e., CRI and R9) for twodifferent types of illuminant: a two-channel light source and afive-channel light source. As shown in FIG. 7, a two-channel lightsource will typically exhibit CRI values of approximately 90 and R9values of approximately 50 across a tunable range of 2700-6500K. Toaccount for the discoloration, significant post-processing must usuallybe performed on images captured in conjunction with flashes produced bythe two-channel light source. Conversely, the five-channel light sourcemay exhibit CRI values and R9 values of approximately 92-98 over atunable range of 1500-8000K.

The five-channel light source may also be optimized for flash events atcertain CCTs. In such embodiments, the five-channel light source couldbe designed to have higher CRI values and/or R9 values than those listedabove in certain ranges (e.g., 4000-5000K, 5000-6000K, etc.).

FIGS. 8A-D illustrate the ability of four different types of lightsource to mimic the visible spectrum of an ambient scene. FIG. 8Acorresponds to conventional flash technology for mobile phones havingcertain operating characteristics (e.g., 5000K CCT, 70 CRI, 0 R9). FIG.8B corresponds to a two-channel light source having a first set ofoperating characteristics (e.g., 5000K CCT, 90 CRI, 50 R9), while FIG.8C corresponds to a two-channel light source having a second set ofoperating characteristics (e.g., 2700K CCT, 90 CRI, 50 R9). FIG. 8D,meanwhile, corresponds to a five-channel light source having certainoperating characteristics (e.g., 97 CRI, 97 R9, 95 Rf). In comparison tothese other light sources, the five-channel light source can moreaccurately reproduce a variety of different colors across a wider rangeof wavelengths, as evidenced by its ability to more closely align withthe reference source.

FIGS. 9A-D illustrate the ability of the four different types of lightsource to properly mimic chromaticity of an ambient scene. Morespecifically, FIGS. 9A-D depict total chromaticity shift across 16different hue bins for each of the four different types of light source.FIG. 9A corresponds to the conventional flash technology for mobilephones. FIG. 9B corresponds to the two-channel light source having thefirst set of operating characteristics. FIG. 9C corresponds to thetwo-channel light source having the second set of operatingcharacteristics. FIG. 9D corresponds to the five-channel light source.In comparing FIGS. 9A-D, those skilled in the art will recognize thatthe five-channel light source can be designed to minimize the averagechromaticity shift across these 16 different hue bins. Said another way,in comparing FIGS. 9A-D, those skilled in the art will recognize thatthe five-channel light source can more accurately reproduce colorsacross the visible spectrum in comparison to these other light sources.

FIG. 10 illustrates the total achievable color gamut 1004 of afive-channel light source in comparison to a conventional chromaticitydiagram 1002. The chromaticity diagram 1002 characterizes colors by aluminance parameter and two color coordinates, which together specify asingle point in the chromaticity diagram 1002. Colors can precisely becompared using the chromaticity diagram 1002 because each parameter isbased on the spectral power distribution (SPD) of the light emitted froma light source and factored by sensitivity curves measured for the humaneye.

In addition to white light of various CCTs, the five-channel lightsource can also be configured to produce colored light by separatelydriving/addressing each color channel. Said another way, thefive-channel light source can produce fully saturated flashes byseparately driving each color channel. Assume, for example, that thefive-channel light source includes five separate color channelsconfigured to produce blue light, cyan light, lime light, amber light,and red light. To produce red light, a controller may causecurrent/voltage to be provided only to the red color channel. Similarly,to produce orange light, the controller may cause current/voltage to beprovided to the red color channel and the amber color channel. Thus, thecontroller may be able to produce light in a variety of different colors(e.g., in accordance with a variety of color mixing models) in additionto white light at a variety of different CCTs.

FIG. 11 illustrates how the five-channel light sources described hereincan substantially improve in terms of color reproducibility incomparison to two-channel light sources. Because a five-channel lightsource can achieve consistently better results across these differentmetrics, an image captured in conjunction with a flash produced by thefive-channel light source will be noticeably better than an imagecaptured in conjunction with a flash produced by a two-channel lightsource. The improved flash may also allow the amount of post-processingto be drastically reduced (e.g., because significant post-processing isno longer needed to address discoloration).

Establishing Effect of Each Color Channel

In some embodiments, a multi-channel light source may automatically tunea flash to produce white light having an appropriate formulation basedon a scene and any other illuminants shining on the scene (e.g.,sunlight, incandescent bulbs, halogen bulbs, fluorescent bulbs). Theeffect of each color channel of a multi-channel light source can beestablished in at least two different ways. While the processesdescribed below involve a five-channel light source, those skilled inthe art will recognize the processes are equally applicable tomulti-channel light sources having any number of channels.

FIG. 12 illustrates a process 1200 for acquiring color information thatmay be useful in tuning each color channel of a five-channel lightsource in preparation for a flash event. The light source can initiallystrobe through each color channel of the five color channels. Morespecifically, the light source can produce a series of discrete coloredflashes by separately driving the illuminant(s) corresponding to a firstcolor channel (step 1201), a second color channel (step 1202), a thirdcolor channel (step 1203), a fourth color channel (step 1204), and afifth color channel (step 1205). Such action will sequentiallyilluminate the scene with colored light of five different colors. Forexample, the scene may be flooded with blue light, cyan light, limelight, amber light, and red light in any order.

By pre-flashing each color channel, valuable information regarding theeffect of flash contribution on image characteristics (e.g., pixellightness, hue, and chromaticity) can be discovered. Each discrete flashcan vary in length from 15 milliseconds to 100 milliseconds. Moreover,each discrete flash may produce approximately 750-800 lumens (e.g., dueto an application of 2-3 amps at 3 volts for 15 milliseconds).

An image sensor may capture a series of images in conjunction with theseries of colored flashed (step 1206). More specifically, the imagesensor may capture at least one image under the colored flash producedby each color channel of the multi-channel light source. Following thesteps 1201-1206, two different data sets will be available to acharacterization module: first data generated by the multi-channel lightsource and second data generated by the image sensor. The first data mayinclude various illuminance characteristics (e.g., driving current,flux, operating temperature, wavelength), while the second data mayinclude image data representing response of the image sensor. The imagesensor may also capture a reference image without any flash (step 1207).

Thereafter, a characterization module can examine the first data and thesecond data. More specifically, the characterization module may examinedata corresponding to the effect of each color channel (step 1208). Toacquire the data, the characterization module may, for a given colorchannel, subtract image data associated with the reference image fromimage data associated with the corresponding image of the series ofimages. For example, if the characterization module would like toestablish the effect of the lime color channel, the characterizationmodule will subtract the reference image from the image taken inconjunction with the lime-colored flash.

Moreover, the characterization module may determine an appropriate lightformulation based on the effect of each color channel (step 1209). Forexample, the characterization module may apply a solver algorithm thatdetermines, based on a reference set of curves corresponding todifferent combinations of driving current and flux, an appropriatebrightness and color point(s) necessary to achieve a particular colormodel. The characterization module and the solver algorithm may residein a memory of the multi-channel light source or a memory of theelectronic device (e.g., mobile phone) in which the multi-channel lightsource is housed. The white light formulation may include a specificoperating parameter for each color channel such that, when the fivecolor channels are illuminated in concert, while light produced by themulti-channel light source will have a particular CCT. Such action mayensure that an image captured in conjunction with the white light willmore accurately replicate how the ambient environment is actually seenby the human eye.

In some embodiments, the characterization module determines anappropriate light formulation corresponding to a flash setting selectedby a user. For example, the characterization module may receive inputindicative of a selection of a flash setting by a user (e.g., via aninterface accessible on the electronic device in which the multi-channellight source resides), and then generate an arbitrary white flash basedon a color mixing model corresponding to the selected flash setting. Asnoted above, the characterization module may alternatively determine theflash setting automatically on behalf of the user based on time,location, etc. For example, in response to determining that the user isattempting to capture an image of a scene at night, the characterizationmodule may identify the appropriate light formulation.

FIG. 13 illustrates another process 1300 for acquiring color informationthat may be useful in tuning each color channel of a five-channel lightsource in preparation for a flash event. Rather than strobe through eachcolor channel separately, the light source can instead produce a seriesof discrete substantially white flashes by driving differentcombinations of channels.

Here, for example, the light source can produce a series of discretesubstantially white flashes by driving the illuminant(s) correspondingto all color channels except the first color channel (step 1301), thesecond color channel (step 1302), the third color channel (step 1303),the fourth color channel (step 1304), and the fifth color channel (step1305). Such action will sequentially illuminate the scene withsubstantially white light, though the tint will differ slightly as eachflash will be missing the illuminant(s) of a single color channel.

An image sensor may capture a series of images in conjunction with theseries of substantially white flashes (step 1306). More specifically,the image sensor may capture at least one image under the substantiallywhite flash produced without each color channel of the multi-channellight source. Following the steps 1301-1306, two different data setswill be available to the characterization module: first data generatedby the multi-channel light source and second data generated by the imagesensor. The first data may include various illuminance characteristics(e.g., driving current, flux, operating temperature, wavelength), whilethe second data may include image data representing response of theimage sensor.

The multi-channel light source may also produce a reference flash byactivating the illuminant(s) corresponding to all color channels (step1307). In such embodiments, the image sensor can capture a referenceimage in conjunction with the reference flash (step 1308). Such actionenables a characterization module to readily determine the effect ofeach color channel without requiring that each color channel beseparately addressed.

Thereafter, the characterization module can examine data correspondingto the effect of each color channel (step 1309). The characterizationcan establish the effect of each color channel via process 1300 of FIG.13 and process 1200 of FIG. 12, though the effect is established indifferent manners. Here, for example, the characterization module may,for a given color channel, subtract image data associated with the imageof the series of images in which the given color channel was notilluminated from the image data associated with the reference image. Forexample, if the characterization module would like to establish theeffect of the lime color channel, the characterization module willsubtract the image taken in conjunction with the substantially whiteflash that does not include lime-colored light from the reference image.

In comparison to the series of colored flashes produced by process 1200of FIG. 12, the series of substantially white flashed produced byprocess 1300 of FIG. 13 may be less objectionable to the human eye.Steps 1309-1301 of FIG. 13 may be substantially identical to steps1208-1209 of FIG. 12.

Those skilled in the art will recognize that other combinations of colorchannels may be illuminated together, so long as the characterizationmodule can ultimately determine the effect of each individual colorchannel.

Unless contrary to physical possibility, it is envisioned that the stepsdescribed above may be performed in various sequences and combinations.For example, the light source may be configured to illuminate all colorchannels before illuminating subsets of color channels (e.g., performsteps 1307-1308 before steps 1301-1306). As another example, the lightsource may be configured to illuminate all color channels between eachflash involving a subset of the channels (e.g., perform steps 1307-1308before step 1302, step 1303, step 1304, and step 1305).

Other steps may also be included in some embodiments. For example, afterdetermining the appropriate white light formulation, the light sourcemay produce a flash that illuminates the ambient environment duringimage capture by an electronic device. More specifically, the lightsource may produce the flash responsive to receiving input indicative ofuser input specifying that an image be captured. The user input may bein the form of tactile input provided along a touch-sensitive display ora mechanical button accessible along the exterior of the electronicdevice. As another example, after determining the appropriate whitelight formulation, the multi-channel light source may produce a seriesof flashes that illuminate the ambient environment during video captureby the electronic device. In such environments, the light source maycommunicate with the electronic device to ensure that the series offlashes temporally align with each video frame captured by theelectronic device.

In some instances, an electronic device may employ “pre-image flashing”to establish the effect of each color channel of a multi-channel lightsource, and then capture an image without any flash. For example, thismay be done for outdoor scenes in which a flash is unnecessary.Pre-image flashing may be formed additively (i.e., in accordance withprocess 1200 of FIG. 12) or subtractively (i.e., in accordance withprocess 1300 of FIG. 13). In some embodiments, the “pre-image” capturedin conjunction with each “pre-image flash” can be compiled into thefinal image. Thus, a separate image may not necessarily be capturedfollowing the series of images captured in conjunction with thepre-image flashes. These pre-images may also be shot at differentshutter speeds, resolutions, etc., than the final image. For example,the electronic device may be configured to capture pre-images at a firstresolution and an additional image at a second resolution. Generally,the first resolution will be lower than the second resolution sincespectral information can be extracted regardless of whether theresolution is sufficient for photography purposes.

Advantages of Multi-Channel Light Sources

Various advantages are enabled when the light sources described hereinare used to produce a flash. Such advantages include:

-   -   Broad-spectrum light having a high gamut area can be created by        mixing the light produced by multiple color channels        corresponding to different colors. By capturing images under the        broad-spectrum light, image quality can be naturally increased        (e.g., in comparison to post-processing the images).    -   By controlling gamut (e.g., by separately addressing each color        channel), the light source can flash specific color channels (or        combinations of color channels) at a high speed to analyze the        relative reflected light of different colors. For example, the        light source may separately flash each color channel before the        actual flash event to determine the white light formulation        (also referred to as the “flash formulation”) that will result        in an image having the highest possible quality. The flash        formulation can be defined by values for characteristics such as        CCT, saturation level, etc. Alternatively, these characteristics        can be pre-manipulated to develop a desired effect.    -   The ability to separately access the gamut of colors producible        by the light source creates opportunities for third parties        (e.g., software developers) to leverage individual color access.        For example, a light source may be configured to blink red        responsive to a determination that the temperature of the        electronic device has exceeded a threshold, the power available        to the electronic device has fallen below a threshold, etc. As        another example, the light source may be configured to blink        blue responsive to a determination that the electronic device is        connected to another electronic device via a short-range        wireless protocol (e.g., Bluetooth®). The light source may be        configured to illuminate in various colors, patterns, etc. Thus,        an individual could select a color via an interface shown on the        display of the electronic device, and then point the light        source toward a white wall to determine the pantone color        desired for painting.    -   Full-spectrum illumination also offers an opportunity to        generate circadian light that may be used, for example, as a        projected source rather than the backlit sources currently        available in many displays (e.g., liquid-crystal displays        (LCDs)).    -   Light sources having multiple color channels (e.g., three        channels, five channels, seven channels, etc.) plus dies can        collectively deliver greater output in lumens than conventional        flash technologies. The additional output may present        opportunities for diffusion in conjunction with mixing.    -   Different geometries can be created that offer better        “roundness” of the light source with matched colors that        complement one another to create symmetrically distributed white        light.

Given that the photography systems of electronic devices are closed(i.e., the strobe spectrum, spectral profiles of the camera sensor, andflash contribution to pixel lightness, hue, and chromaticity are allknown), an imaging apparatus can computationally compensate forlow-fidelity, flash-induced color distortion and then extrapolatemissing color information. The imaging apparatus may include amulti-channel light source, a camera sensor, a controller (e.g., aprocessor), a solver module, or any combination thereof.

Fidelity Comparisons for an Illustrative Example

Several examples of fidelity comparisons between the flash technologyincluded in a series of conventional electronic devices and afive-channel light source having a round, 5,000-lumen array ofilluminants are provided below. Here, each fidelity comparison is basedon a Fujifilm® grey card having RGBCMY color points and a DSC LabsChromaMatch® chart.

FIG. 14 illustrates a process 1400 for performing a processing procedureon images captured by an electronic device. Initially, at least onecolor reference object is imaged by the electronic device in conjunctionwith a flash produced by the built-in flash technology (step 1401).Multiple color reference objects will often be imaged by the electronicdevice to derive additional information on the color properties ofobjects within the ambient environment. Examples of color referenceobjects include color charts, such as the Fujifilm® grey card havingRGBCMY color points and the DSC Labs ChromaMatch® chart.

The at least one color reference object can then be imaged inconjunction with a flash produced by the five-channel light source (step1402). The examples provided below correspond to light produced by alight source having five separately addressable color channels that isseparate from the electronic device. However, the five-channel lightsource could also be incorporated into the electronic device itself(e.g., instead of, or in addition to, the built-in flash technology).

Thereafter, data corresponding to the first image (i.e., the image takenin conjunction with the flash produced by the built-in flash technology)and data corresponding to the second image (i.e., the image taken inconjunction with the flash produced by the five-channel light source)can be examined to detect improvements in color reproducibility (step1403).

FIGS. 15-19 illustrate the improved fidelity of a five-channel lightsource in comparison to conventional flash technology. FIG. 15 depicts afidelity comparison between the five-channel light source and thebuilt-in flash technology of a Huawei® Nexus 6P mobile phone. FIG. 16depicts a fidelity comparison between the five-channel light source andthe built-in flash technology of a Google Pixel™ mobile phone. FIG. 17depicts a fidelity comparison between the five-channel light source andthe built-in flash technology of a Samsung® Galaxy mobile phone. FIG. 18depicts a fidelity comparison between the five-channel light source andthe built-in flash technology of an Apple iPhone® 7 mobile phone. FIG.19 depicts a fidelity comparison between the five-channel light sourceand the built-in flash technology of an Apple iPhone® X mobile phone.

Electronic devices that employ conventional flash technologies relyheavily on internal algorithms to post-process images to create aneutral white light. Post-processing can be highly effective at creatinga neutral white light despite the limited spectrum of the illuminants(e.g., LEDs) used in these conventional flash technologies. However, theresulting color space of the image will be heavily affected bypost-processing, as well as the lack of color information in the source.

To quantify the color space, it is often helpful to look at severaldifferent color points. Here, for example, the color space is defined interms of red (R), green (G), blue (B), cyan (C), magenta (M), and yellow(Y). Red, green, and blue (RGB) are measured with respect to 255distinct points of information. Therefore, a “true” red will be 255/255in the red metric, a “true” green will be 255/255 in the green metric,and a “true” blue will be 255/255 in the blue metric. Cyan, magenta, andyellow (CMY), meanwhile, are measured with respect to two of the threeRGB colors. By comparing the values for RGBCMY, several differentmetrics can be quantified, including the ability of the camera sensor(also referred to as an “image sensor”) to produce a neutral whitelight, the ability to produce a neutral white light duringpost-processing, and the resulting color space followingpost-processing.

In view of FIGS. 15-19, those skilled in the art will recognize that thefive-channel light source consistently produces superior colorrendering, as well as a more complete color gamut, across each referenceelectronic device. Note that some of the electronic devices are morereliant on their flash for white balance than others. For example, theimage sensors in the Huawei® Nexus 6P mobile phone and the Samsung®Galaxy mobile phone appear inferior to the image sensor in the AppleiPhone® 7 mobile phone when a flash is produced by the built-in flashtechnology, but outperform the image sensor in the Apple iPhone® 7mobile phone when a flash is produced by the five-channel light source.Thus, the Apple iPhone® 7 mobile phone appears to be more reliant onpost-processing to create a neutral white light, and the Huawei® Nexus6P mobile phone and the Samsung® Galaxy mobile phone appear to haveinferior built-in flash technology.

Superior image sensors (e.g., those in the Apple iPhone® X mobile phone)can take advantage of greater information about discoloration. Toimprove image quality, these superior image sensors can be paired withimproved processing software designed to overcome the shortcomingsassociated with inferior light sources. However, image quality can befurther improved when images are captured in conjunction with flashesproduced by an improved light source (e.g., a five-channel lightsource).

Processing System

FIG. 20 is a block diagram illustrating an example of a processingsystem 2000 in which at least some operations described herein can beimplemented. For example, some components of the processing system 2000may be hosted on an electronic device that includes a multi-channellight source (e.g., light source 100 of FIGS. 1A-B) or on an electronicdevice that is communicatively connected to a multi-channel light source(e.g., via a cable connection or a wireless connection).

The processing system 2000 may include one or more central processingunits (“processors”) 2002, main memory 2006, non-volatile memory 2010,network adapter 2012 (e.g., network interface), video display 2018,input/output devices 2020, control device 2022 (e.g., keyboard andpointing devices), drive unit 2024 including a storage medium 2026, andsignal generation device 2030 that are communicatively connected to abus 2016. The bus 2016 is illustrated as an abstraction that representsone or more physical buses and/or point-to-point connections that areconnected by appropriate bridges, adapters, or controllers. The bus2016, therefore, can include a system bus, a Peripheral ComponentInterconnect (PCI) bus or PCI-Express bus, a HyperTransport or industrystandard architecture (ISA) bus, a small computer system interface(SCSI) bus, a universal serial bus (USB), IIC (I2C) bus, or an Instituteof Electrical and Electronics Engineers (IEEE) standard 1394 bus (alsoreferred to as “Firewire”).

The processing system 2000 may share a similar computer processorarchitecture as that of a desktop computer, tablet computer, personaldigital assistant (PDA), mobile phone, game console, music player,wearable electronic device (e.g., a watch or fitness tracker),network-connected (“smart”) device (e.g., a television or home assistantdevice), virtual/augmented reality systems (e.g., a head-mounteddisplay), or another electronic device capable of executing a set ofinstructions (sequential or otherwise) that specify action(s) to betaken by the processing system 2000.

While the main memory 2006, non-volatile memory 2010, and storage medium2026 (also called a “machine-readable medium”) are shown to be a singlemedium, the term “machine-readable medium” and “storage medium” shouldbe taken to include a single medium or multiple media (e.g., acentralized/distributed database and/or associated caches and servers)that store one or more sets of instructions 2028. The term“machine-readable medium” and “storage medium” shall also be taken toinclude any medium that is capable of storing, encoding, or carrying aset of instructions for execution by the processing system 2000.

In general, the routines executed to implement the embodiments of thedisclosure may be implemented as part of an operating system or aspecific application, component, program, object, module, or sequence ofinstructions (collectively referred to as “computer programs”). Thecomputer programs typically comprise one or more instructions (e.g.,instructions 2004, 2008, 2028) set at various times in various memoryand storage devices in a computing device. When read and executed by theone or more processors 2002, the instruction(s) cause the processingsystem 2000 to perform operations to execute elements involving thevarious aspects of the disclosure.

Moreover, while embodiments have been described in the context of fullyfunctioning computing devices, those skilled in the art will appreciatethat the various embodiments are capable of being distributed as aprogram product in a variety of forms. The disclosure applies regardlessof the particular type of machine or computer-readable media used toactually effect the distribution.

Further examples of machine-readable storage media, machine-readablemedia, or computer-readable media include recordable-type media such asvolatile and non-volatile memory devices 2010, floppy and otherremovable disks, hard disk drives, optical disks (e.g., Compact DiskRead-Only Memory (CD-ROMS), Digital Versatile Disks (DVDs)), andtransmission-type media such as digital and analog communication links.

The network adapter 2012 enables the processing system 2000 to mediatedata in a network 2014 with an entity that is external to the processingsystem 2000 through any communication protocol supported by theprocessing system 2000 and the external entity. The network adapter 2012can include a network adaptor card, a wireless network interface card, arouter, an access point, a wireless router, a switch, a multilayerswitch, a protocol converter, a gateway, a bridge, bridge router, a hub,a digital media receiver, and/or a repeater.

The network adapter 2012 may include a firewall that governs and/ormanages permission to access/proxy data in a computer network, andtracks varying levels of trust between different machines and/orapplications. The firewall can be any number of modules having anycombination of hardware and/or software components able to enforce apredetermined set of access rights between a particular set of machinesand applications, machines and machines, and/or applications andapplications (e.g., to regulate the flow of traffic and resource sharingbetween these entities). The firewall may additionally manage and/orhave access to an access control list that details permissions includingthe access and operation rights of an object by an individual, amachine, and/or an application, and the circumstances under which thepermission rights stand.

The techniques introduced here can be implemented by programmablecircuitry (e.g., one or more microprocessors), software and/or firmware,special-purpose hardwired (i.e., non-programmable) circuitry, or acombination of such forms. Special-purpose circuitry can be in the formof one or more application-specific integrated circuits (ASICs),programmable logic devices (PLDs), field-programmable gate arrays(FPGAs), etc.

REMARKS

The foregoing description of various embodiments of the claimed subjectmatter has been provided for the purposes of illustration anddescription. It is not intended to be exhaustive or to limit the claimedsubject matter to the precise forms disclosed. Many modifications andvariations will be apparent to one skilled in the art. Embodiments werechosen and described in order to best describe the principles of theinvention and its practical applications, thereby enabling those skilledin the relevant art to understand the claimed subject matter, thevarious embodiments, and the various modifications that are suited tothe particular uses contemplated.

Although the Detailed Description describes certain embodiments and thebest mode contemplated, the technology can be practiced in many ways nomatter how detailed the Detailed Description appears. Embodiments mayvary considerably in their implementation details, while still beingencompassed by the specification. Particular terminology used whendescribing certain features or aspects of various embodiments should notbe taken to imply that the terminology is being redefined herein to berestricted to any specific characteristics, features, or aspects of thetechnology with which that terminology is associated. In general, theterms used in the following claims should not be construed to limit thetechnology to the specific embodiments disclosed in the specification,unless those terms are explicitly defined herein. Accordingly, theactual scope of the technology encompasses not only the disclosedembodiments, but also all equivalent ways of practicing or implementingthe embodiments.

The language used in the specification has been principally selected forreadability and instructional purposes. It may not have been selected todelineate or circumscribe the subject matter. It is therefore intendedthat the scope of the technology be limited not by this DetailedDescription, but rather by any claims that issue on an application basedhereon. Accordingly, the disclosure of various embodiments is intendedto be illustrative, but not limiting, of the scope of the technology asset forth in the following claims.

What is claimed is:
 1. An electronic device comprising: an image sensorconfigured to generate images from light collected through a lens; alight source that includes at least three color channels, wherein eachcolor channel includes one or more illuminants configured to produce asubstantially similar color, and wherein each color channel isseparately addressable to produce light of a different color; a memoryconfigured to store multiple color mixing models associated withmultiple correlated color temperatures (CCTs), wherein each color mixingmodel specifies a driving current to be provided to each color channelof the at least three color channels to achieve the corresponding CCT;and a processor configured to: receive input indicative of a request tocapture an image of an ambient environment viewable through the lens;determine that the image is to be captured in conjunction with a flashproduced by the light source; and generate a signal that promptssimultaneous illumination of each color channel to produce white lighthaving a CCT of at least 1650K and no more than 10000K.
 2. Theelectronic device of claim 1, wherein the light source includes fourcolor channels configured to produce red light, blue light, green light,and amber light.
 3. The electronic device of claim 1, wherein the lightsource includes five color channels configured to produce red light,blue light, amber light, cyan light, and lime light.
 4. The electronicdevice of claim 1, wherein the processor is further configured to:identify, based on a color characteristic of the image sensor or thelight source, a color mixing model from amongst the multiple colormixing models; and determine an operating parameter to drive the atleast three color channels of the light source based on the color mixingmodel.
 5. The electronic device of claim 4, wherein the colorcharacteristic is derived from thermal feedback generated by a heatsensor, optical feedback generated by an optical sensor, or a referenceimage created by the image sensor in conjunction with a reference flashproduced by at least one color channel.
 6. An electronic devicecomprising: an image sensor configured to generate images from lightcollected through a lens; a light source that includes at least threecolor channels, wherein each color channel includes one or moreilluminants configured to produce a substantially similar color, andwherein each color channel is separately addressable to produce light ofa different color; and a processor configured to: receive inputindicative of a request to capture an image of an ambient environmentviewable through the lens; determine that the image is to be captured inconjunction with a flash of white light produced by the light source;cause a series of colored flashes to be produced in which each colorchannel is separately illuminated; cause a series of images to becaptured in conjunction with the series of colored flashes; identify aspectral characteristic of each color channel by examining the image inthe series of images that is captured in conjunction with thecorresponding colored flash; examine the spectral characteristics toestablish a spectral property of the light source; and generate a signalthat prompts simultaneous illumination of each color channel to producethe flash of white light having a correlated color temperature (CCT) ofat least 1650K and no more than 10000K.
 7. The electronic device ofclaim 6, wherein the signal is based on the spectral property of thelight source.
 8. An electronic device comprising: an image sensorconfigured to generate images from light collected through a lens; alight source that includes at least three color channels, wherein eachcolor channel includes one or more illuminants configured to produce asubstantially similar color, and wherein each color channel isseparately addressable to produce light of a different color; and aprocessor configured to: receive input indicative of a request tocapture an image of an ambient environment viewable through the lens;determine that the image is to be captured in conjunction with a flashof white light produced by the light source; cause a reference flash tobe produced in which the at least three color channels aresimultaneously illuminated; cause a reference image to be captured inconjunction with the reference flash; cause a series of flashes to beproduced in which all color channels except one are simultaneouslyilluminated; cause a series of images to be captured in conjunction withthe series of flashes; identify a spectral characteristic of each colorchannel by estimating a corresponding discoloration effect, wherein thecorresponding discoloration effect for a given color channel isestimated by comparing the image in the series of images that iscaptured in conjunction with the flash in which the given color channelwas not illuminated to the reference image; examine the spectralcharacteristics to establish a spectral property of the light source;and generate a signal that prompts simultaneous illumination of eachcolor channel to produce the flash of white light having a correlatedcolor temperature (CCT) of at least 1650K and no more than 10000K. 9.The electronic device of claim 8, wherein the signal is based on thespectral property of the light source.
 10. A light source designed toimprove color reproducibility, the light source comprising: anilluminant array that includes at least three color channels, whereineach color channel includes one or more illuminants configured toproduce a substantially similar color, and wherein each color channel isseparately addressable to produce light of a different color; and acontroller configured to— select a color mixing model corresponding to aparticular correlated color temperature (CCT) from amongst multiplecolor mixing models, and determine, based on the color mixing model, anoperating parameter to simultaneously drive the at least three colorchannels to produce a flash of white light having the particular CCT.11. The light source of claim 10, wherein the illuminant array has atunable CCT range of 1650K to 8000K.
 12. The light source of claim 10,wherein the illuminant array has a diameter of less than fourmillimeters (mm).
 13. The light source of claim 10, further comprising:at least three linear field-effect transistor-based (FET-based) currentregulated drivers, wherein each linear FET-based current regulateddriver is configured to drive a corresponding color channel of the atleast three color channels.
 14. The light source of claim 10, furthercomprising: a heat sensor configured to measure thermal feedback;wherein the controller is further configured to: determine, for eachcolor channel, an effect of the thermal feedback; and adjust the colormixing model responsive to a determination that the effect has exceededa predetermined threshold for at least one color channel.
 15. The lightsource of claim 10, further comprising: multiple optical sensorsconfigured to measure optical feedback at different wavelength ranges;wherein the controller is further configured to: determine, for eachcolor channel, whether color shift has occurred by examining the opticalfeedback; and adjust the color mixing model responsive to adetermination that color shift has exceeded a predetermined thresholdfor at least one color channel.
 16. The light source of claim 10,wherein the controller is further configured to: identify a colorfeature of an image captured by an electronic device in conjunction withthe flash of white light; and adjust the color mixing model based on thecolor feature.
 17. The light source of claim 10, further comprising: acircular substrate to which the illuminant array is mounted, wherein thecircular substrate is comprised of a material able to dissipate heatgenerated by the illuminant array.
 18. A method for producing whitelight with a multi-channel light source having at least three colorchannels, each color channel including one or more illuminantsconfigured to produce a substantially similar color, the methodcomprising: characterizing a spectral property of the multi-channellight source by producing a series of flashes in which each colorchannel is separately illuminated, causing a series of images to becaptured in conjunction with the series of flashes, identifying aspectral characteristic of each color channel by examining the image inthe series of images captured in conjunction with the correspondingflash, and examining the spectral characteristics to discover thespectral property of the multi-channel light source; identifying a colormixing model based on the spectral property; and causing themulti-channel light source to produce white light having a particularcorrelated color temperature (CCT) by separately driving each colorchannel of the at least three color channels in accordance with thecolor mixing model.
 19. A method for producing white light with amulti-channel light source having at least three color channels, eachcolor channel including one or more illuminants configured to produce asubstantially similar color, the method comprising: characterizing aspectral property of the multi-channel light source by producing areference flash in which the at least three color channels aresimultaneously illuminated, causing a reference image to be captured inconjunction with the reference flash, producing a series of flashes inwhich all color channels except one are simultaneously illuminated,causing a series of images to be captured in conjunction with the seriesof flashes, identifying a spectral characteristic of each color channelby estimating a corresponding discoloration effect, wherein thecorresponding discoloration effect for a given color channel isestimated by comparing the image in the series of images that iscaptured in conjunction with the flash in which the given color channelwas not illuminated to the reference image, and examining the spectralcharacteristics to discover the spectral property of the multi-channellight source; identifying a color mixing model based on the spectralproperty; and causing the multi-channel light source to produce whitelight having a particular correlated color temperature (CCT) byseparately driving each color channel of the at least three colorchannels in accordance with the color mixing model.
 20. A methodcomprising: accessing a memory that associates color mixing models withspectral properties of a multi-channel light source, wherein themulti-channel light source includes at least three color channels, eachcolor channel including one or more illuminants configured to produce asubstantially similar color; characterizing a spectral property of themulti-channel light source; identifying an appropriate color mixingmodel in the memory based on the spectral property; and causing themulti-channel light source to produce light having a particularcorrelated color temperature (CCT) by separately driving each colorchannel of the at least three color channels in accordance with theappropriate color mixing model.
 21. The method of claim 20, wherein themulti-channel light source includes at least five color channels. 22.The method of claim 20, wherein the multi-channel light source includesat least seven color channels.
 23. The method of claim 20, wherein themulti-channel light source is housed within an electronic device. 24.The method of claim 23, wherein the memory is accessible to theelectronic device across a network.
 25. The method of claim 23, whereinthe memory is housed within the electronic device.
 26. A light sourcecomprising: a first color channel that includes a first illuminantconfigured to produce red light that is located on a first die; a secondcolor channel that includes a second illuminant configured to produceblue light that is located on a second die; a third color channel thatincludes a third illuminant configured to produce cyan light that islocated on a third die; a fourth color channel that includes a fourthilluminant configured to produce amber light that is located on a fourthdie; a fifth color channel that includes a fifth illuminant configuredto produce lime light that is located on a fifth die; and a controllerconfigured to produce white light having different correlated colortemperatures (CCTs) by controllably distributing electrical currentamongst the first color channel, the second color channel, the thirdcolor channel, the fourth color channel, and the fifth color channel.27. The light source of claim 26, wherein the first color channel, thesecond color channel, the third color channel, the fourth color channel,or the fifth color channel includes multiple illuminants.
 28. The lightsource of claim 26, wherein each illuminant is a light-emitting diode(LED).
 29. The light source of claim 28, wherein the intra-die spacingis no more than 0.1 millimeters (mm).
 30. The light source of claim 26,further comprising: a substrate configured to dissipate heat generatedby the first illuminant, the second illuminant, the third illuminant,the fourth illuminant, and the fifth illuminant.
 31. The light source ofclaim 30, wherein the substrate is comprised of woven fiberglass clothwith an epoxy resin binder.
 32. The light source of claim 30, whereinthe substrate is comprised of ceramic, metal, or any combinationthereof.
 33. The light source of claim 30, further comprising: aprotective cover comprised of a transparent material.